Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Moving from flyteadmin - Flyte webhooks #4147

Closed
wants to merge 39 commits into from
Closed

Moving from flyteadmin - Flyte webhooks #4147

wants to merge 39 commits into from

Conversation

eapolinario
Copy link
Contributor

TL;DR

Add a new webhook processor capable of parsing the workflow event and sending the message to an external service, such as Slack.

  1. Flyteadmin sends a workflow event to SNS
  2. SNS push the event to SQS
  3. FlyteAdmin webhook processor pulls the event from SQS
  4. Parse the event, and then render the payload template ({{ name }} succeeded)
  5. Send the event to external service by calling the webhook endpoint

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

webhook config

externalEvents:
  enable: true
  type: "aws"
  aws:
    region: "us-west-2"
  eventsPublisher:
    topicName: "arn:aws:sns:us-west-2:590375264460:webhook"
    eventTypes:
      - all
webhooknotifications:
  aws:
    region: "us-west-2"
  webhooks:
    name: slack
    url: https://hooks.slack.com/services/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
    payload: "{{ name }} succeeded"
    processor:
      queueName: webhook
      accountId: "590375264460"

Tracking Issue

#2317

Follow-up issue

NA

pingsutw added 30 commits May 19, 2023 16:14
@pingsutw
wip
1fb5f4b 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
0d4bfa6 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
513fbad 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
d334e49 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
add log
d5a48f7 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
update webhook url
126f777 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
cfc33fc 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
c5442dd 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
c75707a 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
a646c2f 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
d071e75 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
a37209f 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
e498539 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
bb05d3c 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
eccd859 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
31d11a4 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
wip
c966a24 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
webhook poc
e0b14d2 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
init
699adce 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
nit
1b3e896 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
More tests
206cefc 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
bump idl
83d7c3a 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
Merged master
bb703aa 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
nit
cd07b2a 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
nit
4343654 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
make generate
96e663d 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
lint
6cee8cb 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
lint
412f8cc 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
Merge branch 'master' of github.com:flyteorg/flyteadmin into webhook-v2
e9ab3cf
@pingsutw
Read webhook url from secret
99c39c0 
Signed-off-by: Kevin Su <[email protected]>
pingsutw and others added 9 commits July 20, 2023 12:46
@pingsutw
add gcp processor
868ce14 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
lint
fcfe202 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
Fixed tests
6330314 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
nit
d6d5343 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
lint
a3828c8 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
merged master
0cd80f9 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
bump idl
b754cf8 
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
bump idl
25c3a0f 
Signed-off-by: Kevin Su <[email protected]>
@eapolinario
Transfer commits
89fba4b 
Signed-off-by: Eduardo Apolinario <[email protected]>
@eapolinario eapolinario mentioned this pull request Oct 3, 2023
Closed
8 tasks
@codecov
Copy link

codecov bot commented Oct 3, 2023 •
edited
Loading

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (d9586b0) 58.98% compared to head (47e98b1) 59.09%.

❗ Current head 47e98b1 differs from pull request most recent head 89fba4b. Consider uploading reports for the commit 89fba4b to get more accurate results

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #4147      +/-   ##
==========================================
+ Coverage   58.98%   59.09%   +0.10%     
==========================================
  Files         619      368     -251     
  Lines       52804    25539   -27265     
==========================================
- Hits        31146    15091   -16055     
+ Misses      19174     9012   -10162     
+ Partials     2484     1436    -1048
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 572 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pingsutw pingsutw closed this Nov 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@eapolinario @pingsutw